Conventional network interfaces for connecting a digital computer to a network generally use one or more queues for controlling transmission and reception of messages to and from the network. There are generally four types of ring queues which are used. A transmit request queue (Txin) has entries which describe buffers of data to transmit. A transmit complete queue (TXdone) has entries which describe buffers which have been transmitted. A free buffer queue (RXin) has entries which describe available buffers in which received data may be deposited. A receive queue (RXdone) has entries which describe buffers filled with received data. There is typically one set of ring queues per network interface connected to the computer.
In a conventional system, to ensure that access between applications on a computer and the network is properly protected, the operating system controls access to these ring queues. For example, an application makes an operating system call to load a transmit request into the transmit request queue. Similarly, an application makes an operating system call to obtain a received buffer. While operating system involvement in handling of the ring queues is a simple solution with general application, it reduces performance because of the overhead involved in invoking operating system activity.
This operating system involvement and resulting overhead can be reduced by using a network interface which allows direct, protected access to applications. Such a network interface has one or more direct application access (DAA) channels for processing transmit and receive requests. Such a network interface is described in "Experiences with a High Speed Network Adapter: A Software Perspective" by P. Druschel, et al. in SIGCOMM '94. A network interface having similar functionality is described in U.S. patent application Ser. No. 08/549,940 filed on Oct. 30, 1995 by Randy B. Osborne et al., entitled NETWORK INTERFACE and assigned to the assignee of this application now U.S. Pat. No. 5,751,951.
A direct application access (DAA) channel uses a translation table to map virtual addresses used by an application to physical addresses which are actually used to access memory. The virtual addresses are found in entries in the ring queues to represent locations of, for example, data to be transmitted, buffers describing transmitted data, free buffers and buffers containing received data. Each DAA channel has at least one fill set of ring queues associated with it. One or more applications are associated with each DAA channel. In an asynchronous transfer mode network, each application may have one or more virtual circuits associated with it.
The transmit side of a DAA channel ensures that messages can be sent into the network only for the virtual circuits assigned to that DAA channel. The receive side of a DAA channel ensures that messages from the network can only be sent to applications associated with the DAA channel on which the messages are received. The network interface maintains pointers to the ring queues for each DAA channel in ring queue registers. To perform to virtual to physical address translation, each DAA channel has a separate two-way associative, variable-sized address translation table implemented using a local memory in the network interface and managed by the operating system.
An important issue is the scalability of the DAA channels to support multiple applications. The benefit of direct application access, namely bypassing the operating system, results from mapping the ring queues directly into application address space. In the prior art the number of DAA channels directly supported by hardware is limited, which in turn limits the number of applications which can simultaneously exploit the direct network access for low latency and high bandwidth. There are ways to improve scalability of the prior art, but they are limited.
To provide low latency access for receiving messages, multiple applications can map the same receive queue into their respective address spaces. One or more pages containing the receive queue can be limited to read-only access so that applications can only read and not modify the queue contents. Any pointers which are read can be interpreted only with the aid of proper address translation, so there is no protection problem with one application reading entries destined for another application. The problem with this scheme is that each application has to poll the receive queue and skip over entries belonging to other applications. The performance impact arising from this polling limits the scalability of this solution. A better solution is to allow an unbounded number of receive queues.
Scalability can also be provided by using multiple sources of free buffers. The prior art limits the source of free buffers to one or two ring queues, in essence two global pools of buffers. In some cases it is convenient to allow more choice in the buffer characteristics, such as size or location, than simply choosing from a global pool. For example, if two virtual circuits are used, one virtual circuit may use credit-based traffic management and another virtual channel simultaneously may use rate-based traffic management. Another way to obtain more flexibility is by using a list of buffers per virtual circuit. A linked list scheme for supporting such a list is described in U.S. patent application Ser. No. 08/422,344, filed Apr. 14, 1995 by Randy B. Osborne, entitled NETWORK INTERFACE HAVING PER-CHANNEL INTERNAL AND EXTERNAL FREE BUFFER LISTS WITH SELF-CLEANING MECHANISMS and assigned to the assignee of this application now U.S. Pat. No. 5,682,553. There are several problems with this scheme, including the difficulty in adding to the linked lists and the lack of an intermediate between global queues and private per virtual circuit queues. For example, linked lists cannot be shared between several virtual circuits. A better solution is to have any number of free buffer queues and to allow the virtual circuits to decide which queue should be used for selecting buffers.
In these systems, to provide the benefits of direct application access to the network, each application has its own DAA channel with its own set of ring queues. Such a design requires duplication of hardware for each application using a DAA channel. Therefore, these systems do not scale well, i.e., these systems do not support efficiently many applications with direct, protected access to the network.